Systems and methods for used learned representations to determine terrain type

ABSTRACT

Disclosed herein are embodiments of systems, methods, and products comprise an analytic server, which provides a terrain segmentation and classification tool for synthetic aperture radar (SAR) imagery. The server accurately segments and classifies terrain types in SAR imagery and automatically adapts to new radar sensors data. The server receives a first SAR imagery and trains an autoencoder based on the first SAR imagery to generate learned representations of the first SAR imagery. The server trains a classifier based on labeled data of the first SAR imagery data to recognize terrain types from the learned representations of the first SAR imagery. The server receives a terrain query for a second SAR imagery. The server translates the second imagery data into the first imagery data and classifies the second SAR imagery terrain types using the classifier trained for the first SAR imagery. By reusing the original classifier, the server improves system efficiency.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of U.S. application Ser.No. 16/012,624, filed Jun. 19, 2018, issuing as U.S. Pat. No.10,719,706, which is incorporated by reference in its entirety.

TECHNICAL FIELD

This application relates generally to methods and systems for advancedimage segmentation for radar imagery.

BACKGROUND

Traditional methods on segmentation and classification of syntheticaperture radar (SAR) imagery may have failed to take advantage ofnuances of the complex data; instead, they have only focused on detectedimagery. As a result, imagery segmentation systems in traditionalmethods require human intervention to clean up boundaries or verifycorrectness. Since segmentation is a prerequisite for terrainclassification, these early segmentation errors propagate downstream.Consumers of classified terrain regions, such as automated targetrecognition (ATR), may miss crucial search areas. The traditionalmethods may not be resilient against noisy or distorted inputs.

Furthermore, the traditional methods of segmentation and classificationof SAR imagery may not be adaptable to new radar instrumentation. It iscommon for machine learning systems to require days or weeks of trainingbefore beginning to produce accurate results. Once the system has beentrained on the data from one sensor, adapting to a new sensor willcommonly require another large batch of labeled training data. Dependingon circumstances, this training data may not exist or may beprohibitively expensive.

SUMMARY

What is therefore desired is to have a fast, accurate, and adaptableradar imagery segmentation and classification system in order to enhanceintelligence-gathering capability and improve efficiency. Embodimentsdisclosed herein solve the aforementioned problems and other problems bydeveloping Nested Autoencoding of Radar for Neural Image Analysis(NARNIA), a terrain segmentation and classification tool for SAR imagerybased on deep learning neural networks with an innovative nestedautoencoder structure. The NARNIA system applies neural networks to theproblem of identifying terrain types in SAR, a novel arrangement ofautoencoders enables NARNIA to quickly and cheaply adapt to data from anew sensor system. The NARNIA system may filter out irrelevant imageryfrom different types of radar sensors and select only the imagery thatis relevant to an Automated Target Recognition (ATR) search.

In one embodiment, a computer implemented method comprises receiving, bya computer, a first imagery from a first sensor device, wherein thefirst imagery comprises an unlabeled dataset containing original imagerydata obtained by the first sensor device; training, by the computer, anautoencoder by performing unsupervised learning on the unlabeled datasetof the first imagery to generate learned representations of the firstimagery; training, by the computer, a classifier by performingsupervised learning on a labeled dataset of the first imagery, whereinthe labeled dataset comprises terrain types data of the first imagery,wherein the classifier is configured to determine terrain types based onthe learned representations of the first imagery; translating, by thecomputer, a second imagery from a second sensor device into the learnedrepresentations of the first imagery; determining, by the computer,terrain types in the second imagery using the classifier trained for thefirst imagery, wherein the classifier segments and classifies the secondimagery based on the translated learned representations of the firstimagery; and displaying, by the computer, the terrain types in thesecond imagery on a graphical user interface.

In another embodiment, a system comprises a first sensor device; asecond sensor device; a server in communication with the first andsecond sensor devices and configured to: receive a first imagery fromthe first sensor device, wherein the first imagery comprises anunlabeled dataset containing original imagery data obtained by the firstsensor device; train an autoencoder by performing unsupervised learningon the unlabeled dataset of the first imagery to generate learnedrepresentations of the first imagery; train a classifier by performingsupervised learning on a labeled dataset of the first imagery, whereinthe labeled dataset comprise terrain types data of the first imagery,wherein the classifier is configured to determine terrain types based onthe learned representations of the first imagery; translate a secondimagery from a second sensor device into the learned representations ofthe first imagery; determine terrain types in the second imagery usingthe classifier trained for the first imagery, wherein the classifiersegments and classifies the second imagery based on the translatedlearned representations of the first imagery; and display the terraintypes in the second imagery on a graphical user interface.

It is to be understood that both the foregoing general description andthe following detailed description are exemplary and explanatory and areintended to provide further explanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings constitute a part of this specification andillustrate embodiments of the subject matter disclosed herein.

FIG. 1 illustrates a computer system for image segmentation andclassification for radar imagery, according to an embodiment.

FIG. 2 illustrates a workflow for filtering SAR imagery for specificterrain types with NARNIA, according to an embodiment.

FIG. 3 illustrates a flowchart depicting operational steps for imagesegmentation and classification for radar imagery, according to anembodiment.

FIG. 4 illustrates an example of autoencoder performing unsupervisedlearning to generate learned representations, according to anembodiment.

FIG. 5 illustrates an example of classifier based on supervised learningthat maps learned representations to class values, according to anembodiment.

FIG. 6 illustrates an example of a nested autoencoder translating newsensor data into the learned representations of the originalinstrumentation, according to an embodiment.

FIG. 7 illustrates an example of a first sensor data marked withreference objects/scenes, according to an embodiment.

FIG. 8 illustrates an example of a second sensor data marked withreference objects/scenes, according to an embodiment.

FIG. 9 illustrates an example of translating the second sensor data intothe first sensor data based on reference objects/scenes, according to anembodiment.

FIG. 10 illustrates an example of training an artificial intelligencemodel or artificial neural networks (ANN) model for the first sensor toidentify terrain types, according to an embodiment.

FIG. 11 illustrates an example of identifying terrain types in thesecond sensor data of real world objects/scenes, according to anembodiment.

FIG. 12 illustrates an example of identified terrain types in the firstsensor data, according to an embodiment.

FIG. 13 illustrates an example of identified terrain types in the secondsensor data, according to an embodiment.

DETAILED DESCRIPTION

Reference will now be made to the illustrative embodiments illustratedin the drawings, and specific language will be used here to describe thesame. It will nevertheless be understood that no limitation of the scopeof the claims or this disclosure is thereby intended. Alterations andfurther modifications of the inventive features illustrated herein, andadditional applications of the principles of the subject matterillustrated herein, which would occur to one ordinarily skilled in therelevant art and having possession of this disclosure, are to beconsidered within the scope of the subject matter disclosed herein. Thepresent disclosure is here described in detail with reference toembodiments illustrated in the drawings, which form a part here. Otherembodiments may be used and/or other changes may be made withoutdeparting from the spirit or scope of the present disclosure. Theillustrative embodiments described in the detailed description are notmeant to be limiting of the subject matter presented here.

The analytic server develops nested autoencoding of radar for neuralimage analysis (NARNIA). By executing NARNIA, the analytic serverprovides an interface for filtering SAR imagery by terrain type and aterrain segmentation and classification tool for synthesis apertureradar (SAR) imagery. The analytic server accurately segments andclassifies terrain in SAR imagery and can automatically adapt to newradar sensors with unlabeled training data. As an imagery filter, NARNIAcan improve the efficiency of targeted systems by eliminating irrelevantdata in a first path. Possible applications include intelligence,surveillance, reconnaissance (ISR), automated target recognition (ATR),attention focusing systems, and automated detection systems.

The analytic server receives a first SAR imagery and trains anautoencoder based on the first SAR imagery. The analytic server uses theautoencoder to generate learned representations of the first SAR imageryfor the purpose of dimensionality reduction. The analytic server trainsa classifier based on labeled data of the first SAR imagery data torecognize specific terrain types from the learned representations of thefirst SAR imagery. The analytic server receives a terrain query for asecond SAR imagery from a client computing device. The analytic servertranslates the second imagery data into the first imagery data. Theanalytic server segments and classifies the second SAR imagery using theclassifier trained using labeled data on the first SAR imagery. Byreusing the original terrain classifier trained for first sensor todetermine the terrain types of second sensor, the analytic serverimproves the system efficiency.

FIG. 1 illustrates components of a system 100 for image segmentation forradar imagery, according to an embodiment. The system 100 may include ananalytic server 110 with a database 120, a set of client computingdevices 130. The analytic server 110 may connect with the clientcomputing devices 130 via hardware and software components of one ormore networks 140. Examples of the network 140 include, but are notlimited to, Local Area Network (LAN), Wireless Local Area Network(WLAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), andthe Internet. The communication over the network 140 may be performed inaccordance with various communication protocols, such as TransmissionControl Protocol and Internet Protocol (TCP/IP), User Datagram Protocol(UDP), and IEEE communication protocols.

The analytic server 110 may be any computing device comprising aprocessor and other computing hardware and software components,configured to build NARNIA. The analytic server 110 may be logically andphysically organized within the same or different devices or structures,and may be distributed across any number of physical structures andlocations (e.g., cabinets, rooms, buildings, cities). The analyticalserver 110 may receive a request from the client computing device 130 tosegment and classify SAR imagery. The analytic server 110 may developnested autoencoding of radar for neural image analysis (NARNIA), aterrain segmentation and classification tool for SAR imagery. Theanalytic server 110 may train an artificial intelligence model thatcomprises the NARNIA tool. Specifically, the analytic server 110 maydevelop NARNIA based on deep learning neural networks with an innovativenested autoencoder structure. The analytic server 110 may train theartificial intelligence model or develop the NARNIA in two phases. Inthe first phase, the analytic server 110 may train an autoencoder byperforming unsupervised learning. The autoencoder may create reduceddimensionality representations, called learned representations, of theinput SAR data. In the second phase, the analytic server 110 may train aclassifier by performing supervised learning to recognize specificterrain types from the learned representations. In the supervisedlearning step, the analytic server 110 may use training data withlabeled terrain segments, train a classifier to map the learnedrepresentations to class values (e.g., terrain types). The analyticserver 110 may automatically adapt to varying image quality and otherchallenges and may be adaptable to a radar imagery from a variety ofsystems.

The database 120 may be any non-transitory machine-readable mediaassociated with the analytic server 110. The database 120 may beconfigured to store data, including input SAR imagery data, anartificial intelligence model comprising an autoencoder for unsupervisedlearning and a classifier based on supervised learning, unlabeled data,labeled data. The database 120 may also include any other intermediatedata, such as learned representations, for the artificial intelligencemodel.

The client computing device 130 may be any computing device allowing aclient to interact with the analytic server 110. The client computingdevice 130 may be any computing device comprising a processor andnon-transitory machine-readable storage medium. The examples of thecomputing device may include, but are not limited to, a desktopcomputer, a laptop, a personal digital assistant (PDA), a smartphone, atablet computer, and the like. The client computing device 130 maycomprise any number of input and output devices supporting various typesof data, such as text, image, audio, video, and the like. The clientcomputing device 130 may issue a request to segment and classify one ormore SAR imageries, and transmit relevant inputs and user parameters tothe analytic server 110.

FIG. 2 illustrates a workflow for filtering SAR imagery for specificterrain types with NARNIA, according to an embodiment. A user 202operating the client computing device may issue a request comprising aSAR imagery 204 and a terrain query 206 that requests the analyticserver to segment and classify terrain types in the SAR imagery. Inoperation, the analytic server may provide and display an interactiveuser interface on the client computing device. The user interface mayallow the user to upload the one or more SAR imageries for terrainclassification query. After the analytic server receives the query fromthe client computing device, the analytic server may apply the NARNIA208 tool on the received SAR imagery, segment and classify differenttypes of terrain in the SAR imagery, and output the filtered SAR imagery210. The analytic server may display the output filtered SAR imagery onthe client computing device. By filtering the imagery, the analyticserver may improve the accuracy and efficiency of more intensiveoperations, such as automated target detection algorithm searching forcars in urban areas.

FIG. 3 illustrates a flowchart depicting operational steps for imagesegmentation and classification for radar imagery, according to anembodiment. Other embodiments may comprise additional or alternativesteps, or may omit some steps altogether.

At step 302, the analytic server may receive a first SAR imagery fromthe client computing device or retrieve the first SAR imagery from thedatabase. The first SAR imagery input may be taken by a radar sensor,for example, sensor S. The analytic server may use the first SAR imagerydata to train the NARNIA tool, which is an artificial intelligence modelused to segment and classify new SAR imageries. The first SAR imagerymay comprise unlabeled input SAR dataset.

At step 304, the analytic server may train an autoencoder and generatelearned representations of the first SAR imagery using the autoencoder.The autoencoders are a common type of neural network that is trained tomake its output match its input. Critically, there is a hidden layerwith fewer dimensions in the output than the input. The output is calledlearned representations. The autoencoder approach to dimensionalityreduction is capable of making non-linear discriminations, providedthere are a sufficient number of hidden layers. The analytic server mayapply the autoencoder to perform unsupervised learning over unlabeledinput SAR data (e.g., the first SAR imagery data from sensor S).Unlabeled training data is simply data obtained by using the radarsensors, which is relatively easy to gather. The autoencoder maygenerate a reduced dimensionality representation, learnedrepresentations, of the input SAR data. FIG. 4 illustrates an example ofautoencoder performing unsupervised learning to generate learnedrepresentations.

At step 306, the analytic server may train a classifier based on labeleddata of the first SAR imagery data. The analytic server may use theclassifier to recognize specific terrain types from the learnedrepresentations of the first SAR imagery. The analytic server mayconduct a supervised learning to train the classifier using trainingdata with labeled terrain segments. In other words, the analytic servermay require a batch of labeled training data on the first SAR imagery'sterrain segments to train the classifier. The first imagery may comprisea labeled training dataset that labels the terrain types in the firstimagery. The classifier may map the learned representations to classvalues. The class values may be the returned terrain types, such asroad, forest, agricultural area, urban area, rural area, and the like.The classifier may user a deep neural network with many hidden layers.The learned representations may have many more dimensions than thenumber of terrain classes. Training of the classifier may require largeamounts of labeled training data in order to reach its full potential.FIG. 5 illustrates an example of classifier based on supervised learningthat maps learned representations to class values.

At step 308, the analytic server may receive a terrain query for asecond SAR imagery from the client computing device. The second SARimagery data may be from a different sensor, for example, sensor T.Different radar sensors may have images with varying image qualityand/or different environmental conditions. Once the system has beentrained on the data from one sensor, adapting to a new sensor maycommonly require another large batch of labeled training data. Dependingon circumstances, such training data may not exist or may beprohibitively expensive. To improve the system efficiency and reduce thecomputing cost for SAR data from new sensor types, the analytic servermay use unsupervised learning to adapt to new types of radar sensorswhile reusing the original terrain classifier and effectively transferthe preexisting knowledge about SAR imagery to the new sensor type.

At step 310, the analytic server may map or translate the second SARimagery data into the learned representations of the first SAR imagerydata. To grant the ability to adapt to new types of radar sensors (e.g.,from sensor S to new sensor T) rapidly without requiring labeledtraining data, the analytic server may perform unsupervised learning onunlabeled training data of sensor T's SAR data.

In some embodiments, the analytic server may use the autoencoder forsensor S (e.g., S autoencoder) and place the autoencoder for sensor S inthe middle of a new autoencoder for sensor T (e.g., T autoencoder). Theweight of the S autoencoder are fixed to their original values. Theanalytic server may train the T autoencoder to reproduce the input forthe S autoencoder. In this step, the analytic server may employ theneural network to learn how to cast T in terms of S's learnedrepresentations. If there are only linear differences between the datareturned by sensors S and T, the analytic server may need only a singlelayer, which enables this step to be performed much more rapidly thanthe original training. After translating T data into learnedrepresentations of S data, the analytic server may reuse the originalterrain classifier, effectively transferring the preexisting knowledgeabout SAR imagery to the new radar sensor (e.g., sensor T) type. FIG. 6illustrates an example of translating new sensor data into the learnedrepresentations of the original instrumentation.

In some other embodiments, the analytic server may use reference objectsor reference scenes to translate the second SAR imagery data (T data)into the learned representations of the first SAR imagery data (S data).Specifically, the analytic server may require the S data and T data tobe marked with reference objects/scenes. The reference objects/scenesmay be data easily obtained or already available data to indicate theobjects/scenes included in the imageries. For example, the referenceobjects/scenes may be a Ford Taurus® scene, a golf course turf scene, atest center scene, or any other scene. With the reference objects/scenesavailable, the SAR imagery data may be categorized into several groups.The analytic server may determine the correspondence relationshipbetween the S data groups and the T data groups based on the referenceobjects/scenes. Furthermore, the analytic server may translate the Tdata into S data by referring to the correspondence relationship. FIG. 7illustrates an example of the S data marked with referenceobjects/scenes. FIG. 8 illustrates an example of the T data marked withreference objects/scenes. FIG. 9 illustrates an example of translating Tdata into S data based on reference objects/scenes.

Adding support for the new radar sensor T is fast and automatic, givenunlabeled training data. In essence, the analytic server may wrap anautoencoder for the new sensor around the original autoencoder for S.The analytic server may perform unsupervised learning on the outerautoencoder while keeping the original autoencoder's weights fixed. Theresulting neural network may quickly learn to translate T sensor datainto learned representations of the original S sensor. Removing the lasthalf of the neural network yields a T-to-learned representationstranslator, enabling direct use of the original classifier withoutretraining it.

At step 312, the analytic server may segment and classify the second SARimagery using the classifier trained using labeled data on the first SARimagery. As discussed above, the analytic server may translate the Tdata into the learned representations of the original S data. Theanalytic server may use the original classifier trained based on the Sdata to segment and classify the T data and determine the class values,such as the terrain types of the second SAR imagery. The analytic servermay output and display the terrain types of the second imagery on agraphical user interface of the client computing device.

FIG. 4 illustrates an example of autoencoder 400 performing unsupervisedlearning to generate learned representations, according to anembodiment. The autoencoder may take SAR imagery as SAR input 402 forthe autoencoder and perform unsupervised learning over the SAR input402. The autoencoder may generate and output reduced dimensionalityrepresentations, learned representations 404. The SAR input may beunlabeled data. The output learned representations may be in lessdimensions than the input. The autoencoder approach to dimensionalityreduction is capable of making non-linear discriminations, providedthere are a sufficient number of hidden layers.

FIG. 5 illustrates an example of classifier 500 based on supervisedlearning that maps learned representations to class values, according toan embodiment. The analytic server may perform a supervised learningstep to train the classifier using training data with labeled terrainsegments. In other words, the analytic server may require a batch oflabeled training data on the first SAR imagery's terrain segments totrain the classifier. The classifier may recognize specific terraintypes from the learned representations 502 and output class values inclass layer 504. The class values may be the recognized terrain types,such as road, forest, agriculture area, urban area, rural area, and thelike.

FIG. 6 illustrates an example of a nested autoencoder 600 translatingnew sensor data into the learned representations of the originalinstrumentation, according to an embodiment. The analytic server mayprovide the ability to adapt to new types of radar sensors (e.g., sensorT) by translating the new sensor data into the learned representationsof the original sensor (e.g., sensor S) with nested autoencoder. Forexample, the nested autoencoder may perform unsupervised learning onunlabeled training data of new sensor's SAR input 602 (e.g., T's SARdata). The analytic server may train the T autoencoder to reproduce theinput for the S autoencoder 604. The analytic server may employ theneural network to learn how to cast T in terms of S's learnedrepresentations 606. After translating T data into learnedrepresentations of S data, the analytic server may reuse the originalterrain classifier, effectively transferring the preexisting knowledgeabout SAR imagery to the new radar sensor type.

FIG. 7 illustrates an example of the first sensor data (S data) markedwith reference objects/scenes 700, according to an embodiment. The SARimagery from sensor S 702 may be marked with reference objects/scenes704 a, 704 b, 704 c. The analytic server may categorize the S data intodifferent groups with each group corresponding to one reference object.For example, reference data 706 a, including S11, S12, S13, and S14, maycorrespond to the first reference object/scene 704 a. Similarly,reference data 706 b, including S21, S22, S23, and S24, may correspondto the second reference object/scene 704 b. Reference data 706 c,including S31, S32, S33, and S34, may correspond to the third referenceobject/scene 704 c.

FIG. 8 illustrates an example of the second sensor data (T data) markedwith reference objects/scenes 800, according to an embodiment. The SARimagery from sensor T 802 may be marked with reference objects/scenes804 a, 804 b, 804 c. The analytic server may categorize the T data intodifferent groups with each group corresponding to one reference object.For example, reference data 806 a, including T11, T12, T13, and T14, maycorrespond to the first reference object/scene 804 a. Similarly,reference data 806 b, including T21, T22, T23, and T24, may correspondto the second reference object/scene 804 b. Reference data 806 c,including T31, T32, T33, and T34, may correspond to the third referenceobject/scene 804 c.

FIG. 9 illustrates an example of translating the second sensor data (Tdata) into the first sensor data (S data) based on referenceobjects/scenes 900, according to an embodiment. The T data (Tx1, Tx2,Tx3, Tx4, x=1, 2, 3) 902 may correspond to reference objects/scenes 904;the S data (Sx1, Sx2, Sx3, Sx4, x=1, 2, 3) 908 may correspond toreference objects/scenes 906. The reference objects/scenes 904, 906 maybe the same scenes, thus providing the analytic server with informationon which group of T data correspond with which group of S data. Theanalytic server may translate the T data 902 into the S data 908 basedon the reference objects/scenes 904, 906.

FIG. 10 illustrates an example of training an artificial intelligencemodel or artificial neural networks (ANN) model for the first sensor(sensor S) to identify terrain types 1000, according to an embodiment.The analytic server may train the terrain classification ANN model basedon labeled training data available for desired terrain types to beclassified. The analytic server may use the SAR imagery data from sensorS 1002 as input and generate learned representations 1004. The analyticserver may train the classifier 1006 using the training data withlabeled terrain segments. The classifier may recognize the specificterrain types from the learned representations 1004. The terrain typesmay include road, forest, agriculture, urban, rural, and the like.

FIG. 11 illustrates an example of identifying terrain types in thesecond sensor data of real world objects/scenes 1100, according to anembodiment. The analytic server may identify terrain types in the secondsensor data by reusing the original terrain classifier trained for thefirst sensor (sensor S) without requiring SAR data to have comprehensiveterrain labels for the second sensor (sensor T). Specifically, theanalytic server may translate the sensor T's data 1102 to the sensor S'sdata 1108 based on their reference objects/scenes 1104, 1106. The sensorT's reference objects/scenes 1104 and the sensor S's referenceobjects/scenes 1106 may be the same. The analytic server may reuse theoriginal terrain classifier 1112 trained for sensor S to determine theterrain type of sensor T's data. The original terrain classifier mayrecognize the specific terrain types from the learned representations1110.

FIG. 12 illustrates an example of identified terrain types in the firstsensor data 1200, according to an embodiment. The analytic server mayanalyze the terrain-labeled learned representations of sensor S data tolearn the number of terrain classes, class cluster centroid, andvariance by class. In FIG. 12, the learned representations may have twodimensions, such as LR1 1202 and LR2 1204. The terrain types may havefive classes, such as road 1206, forest 1208, agriculture 1210, urban1212, rural 1214.

FIG. 13 illustrates an example of identified terrain types in the secondsensor data 1300, according to an embodiment. The analytic server mayuse the clustering information from the first sensor's (sensor S's) datato efficiently test hypotheses about possible correspondences betweensensor S and sensor T learned representations. FIG. 13 shows an exampleindicating one clustering hypothesis for a given sensor T dataset. Theanalytic server may test the hypothesis by automatically labelingterrain types in radar images from sensor T and having an expert quicklyperform a visual evaluation of the labeling accuracy. If the accuracy islow (e.g., smaller than a threshold value), the analytic server may testadditional clustering hypotheses until the accuracy is sufficient (e.g.,satisfying the threshold value). As the analytic server learns moreinner-sensor correspondence relationships, the analytic server may findtrends and make better guesses about the correct clustering hypothesisfor new sensor data. As shown in the figure, the analytic server mayclassify the radar image from sensor T based on two dimensions, such asLR1 1302 and LR2 1304, and segment the radar image as five terraintypes, such as road 1306, forest 1308, agriculture 1310, urban 1312,rural 1314, with one learned representation not identified 1316 andmarked as unknown.

The foregoing method descriptions and the process flow diagrams areprovided merely as illustrative examples and are not intended to requireor imply that the steps of the various embodiments must be performed inthe order presented. The steps in the foregoing embodiments may beperformed in any order. Words such as “then,” “next,” etc. are notintended to limit the order of the steps; these words are simply used toguide the reader through the description of the methods. Althoughprocess flow diagrams may describe the operations as a sequentialprocess, many of the operations can be performed in parallel orconcurrently. In addition, the order of the operations may bere-arranged. A process may correspond to a method, a function, aprocedure, a subroutine, a subprogram, and the like. When a processcorresponds to a function, the process termination may correspond to areturn of the function to a calling function or a main function.

The various illustrative logical blocks, modules, circuits, andalgorithm steps described in connection with the embodiments disclosedherein may be implemented as electronic hardware, computer software, orcombinations of both. To clearly illustrate this interchangeability ofhardware and software, various illustrative components, blocks, modules,circuits, and steps have been described above generally in terms oftheir functionality. Whether such functionality is implemented ashardware or software depends upon the particular application and designconstraints imposed on the overall system. Skilled artisans mayimplement the described functionality in varying ways for eachparticular application, but such implementation decisions should not beinterpreted as causing a departure from the scope of this disclosure orthe claims.

Embodiments implemented in computer software may be implemented insoftware, firmware, middleware, microcode, hardware descriptionlanguages, or any combination thereof. A code segment ormachine-executable instructions may represent a procedure, a function, asubprogram, a program, a routine, a subroutine, a module, a softwarepackage, a class, or any combination of instructions, data structures,or program statements. A code segment may be coupled to another codesegment or a hardware circuit by passing and/or receiving information,data, arguments, parameters, or memory contents. Information, arguments,parameters, data, etc. may be passed, forwarded, or transmitted via anysuitable means including memory sharing, message passing, token passing,network transmission, etc.

The actual software code or specialized control hardware used toimplement these systems and methods is not limiting of the claimedfeatures or this disclosure. Thus, the operation and behavior of thesystems and methods were described without reference to the specificsoftware code being understood that software and control hardware can bedesigned to implement the systems and methods based on the descriptionherein.

When implemented in software, the functions may be stored as one or moreinstructions or code on a non-transitory computer-readable orprocessor-readable storage medium. The steps of a method or algorithmdisclosed herein may be embodied in a processor-executable softwaremodule, which may reside on a computer-readable or processor-readablestorage medium. A non-transitory computer-readable or processor-readablemedia includes both computer storage media and tangible storage mediathat facilitate transfer of a computer program from one place toanother. A non-transitory processor-readable storage media may be anyavailable media that may be accessed by a computer. By way of example,and not limitation, such non-transitory processor-readable media maycomprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage,magnetic disk storage or other magnetic storage devices, or any othertangible storage medium that may be used to store desired program codein the form of instructions or data structures and that may be accessedby a computer or processor. Disk and disc, as used herein, includecompact disc (CD), laser disc, optical disc, digital versatile disc(DVD), floppy disk, and Blu-ray disc where disks usually reproduce datamagnetically, while discs reproduce data optically with lasers.Combinations of the above should also be included within the scope ofcomputer-readable media. Additionally, the operations of a method oralgorithm may reside as one or any combination or set of codes and/orinstructions on a non-transitory processor-readable medium and/orcomputer-readable medium, which may be incorporated into a computerprogram product.

The preceding description of the disclosed embodiments is provided toenable any person skilled in the art to make or use the embodimentsdescribed herein and variations thereof. Various modifications to theseembodiments will be readily apparent to those skilled in the art, andthe generic principles defined herein may be applied to otherembodiments without departing from the spirit or scope of the subjectmatter disclosed herein. Thus, the present disclosure is not intended tobe limited to the embodiments shown herein but is to be accorded thewidest scope consistent with the following claims and the principles andnovel features disclosed herein.

While various aspects and embodiments have been disclosed, other aspectsand embodiments are contemplated. The various aspects and embodimentsdisclosed are for purposes of illustration and are not intended to belimiting, with the true scope and spirit being indicated by thefollowing claims.

What is claimed is:
 1. A computer-implemented method comprising: receiving, by a computer, from a client computer a request including a query imagery generated by a sensor device of a first device type; translating, by the computer, the query imagery from the first device type to a learned representation for a second device type by applying an autoencoder on the query imagery, the autoencoder trained to generate learned representations of the second device type; identifying, by the computer, one or more terrain types in the learned representation of the query imagery by applying a classifier on the learned representation, the classifier trained to determine the one or more terrain types based upon one or more learned representations of the second device type; and generating, by the computer, a graphical user interface to display the one or more terrain types in the learned representation at the client computer.
 2. The method according to claim 1, further comprising training, by the computer, the autoencoder by performing unsupervised learning on a unlabeled dataset of a training imagery of the second device type to generate learned representations of the a training imagery.
 3. The method according to claim 2, wherein the computer trains a new autoencoder in response to receiving a new training imagery of a new device type.
 4. The method according to claim 2, wherein the training imagery comprises the unlabeled dataset containing original imagery data obtained by a second sensor device of the second device type.
 5. The method according to claim 1, further comprising training, by the computer, the classifier by performing supervised learning on a labeled dataset of a training imagery of the second device type, wherein the labeled dataset comprises the one or more terrain types of the training imagery.
 6. The method according to claim 5, further comprising receiving, by the computer, the labeled dataset of the training imagery of the second device type.
 7. The method according to claim 5, wherein the labeled dataset includes one or more reference objects corresponding to the one or more terrain types.
 8. The method according to claim 7, wherein the reference objects provide a correspondence relationship between the training imagery and the query imagery.
 9. The method according to claim 5, further comprising receiving, by a computer, the training imagery from one or more sensor devices of the second device type.
 10. The method according to claim 1, further comprising retrieving, by the computer, from a database the classifier trained for the second device type.
 11. A system comprising: a first sensor device of a first device type; a second sensor device of a second device type; and a server comprising a processor configured to: receive from a client computer a request including a query imagery generated by a sensor device of a first device type; translate the query imagery from the first device type to a learned representation for a second device type by applying an autoencoder on the query imagery, the autoencoder trained to generate learned representations of the second device type; identify one or more terrain types in the learned representation of the query imagery by applying a classifier on the learned representation, the classifier trained to determine the one or more terrain types based upon one or more learned representations of the second device type; and generate a graphical user interface to display the one or more terrain types in the learned representation at the client computer.
 12. The system according to claim 11, wherein the server is further configured train the autoencoder by performing unsupervised learning on a unlabeled dataset of a training imagery of the second device type to generate learned representations of the a training imagery.
 13. The system according to claim 12, further comprising a new sensor device of a new device type, wherein the server trains a new autoencoder in response to receiving the new training imagery of the new device type.
 14. The system according to claim 12, wherein the training imagery comprises the unlabeled dataset containing original imagery data obtained by the second sensor device of the second device type.
 15. The system according to claim 11, wherein the server is further configured to train the classifier by performing supervised learning on a labeled dataset of a training imagery of the second device type, wherein the labeled dataset comprises the one or more terrain types of the training imagery.
 16. The system according to claim 15, wherein the server is further configure to receive the labeled dataset of the training imagery of the second device type.
 17. The system according to claim 15, wherein the labeled dataset includes one or more reference objects corresponding to the one or more terrain types.
 18. The system according to claim 17, wherein the reference objects provide a correspondence relationship between the training imagery and the query imagery.
 19. The system according to claim 15, wherein the server is further configure to receive the training imagery from one or more sensor devices of the second device type.
 20. The system according to claim 11, further comprising a processor configured to store one or more classifiers, wherein the server is further configure to retrieve from the database the classifier trained for the second device type. 